Convergence results for projected line-search methods on varieties of low-rank matrices via \L{}ojasiewicz inequality

As an intial step towards low-rank optimization algorithms using hierarchical tensors, the aim of this paper is to derive convergence results for projected line-search methods on the real-algebraic variety M≤k of real m × n matrices of rank at most k. Such methods extend successfully used Riemannian optimization methods on the smooth manifold Mk of rank-k matrices to its closure by taking steps along gradient-related directions in the tangent cone, and afterwards projecting back to M≤k. Considering such a method circumvents the difficulties which arise from the non-closedness and the unbounded curvature of Mk. The point-wise convergence is obtained for real-analytic functions on the basis of a Lojasiewicz inequality for the projection of the negative gradient to the tangent cone. If the derived limit point lies on the smooth part of M≤k, i.e. in Mk, this boils down to more or less known results, but with the benefit that asymptotic convergence rate estimates (for specific step-sizes) can be obtained without an a-priori curvature bound, simply from the fact that the limit lies on a smooth manifold. At the same time, one can give a convincing justification for assuming critical points to lie in Mk: if X is a crtitical point of f on M≤k, then either X has rank k, or ∇f(X) = 0.